It has become increasingly common for users to expect the most relevant, up-to-date information to be provided via the Internet. Typically, users enter a search query and are presented with, what a search engine hopes is, the most relevant information responsive to the user's search query. Many search engines rely on “Web crawlers” to gather Web page data in order to provide the data to users.
A search engine may desire to include social networking content in the data scoured by the Web crawler. The content associated with social networking environments should be, as any other content presented to a user, relevant and recent. To ensure that relevant data is available to a user quickly, crawl lists are utilized to organize content that is to be gathered by the Web crawler such that more relevant information is crawled prior to less relevant information. Content associated with social networking environments is, however, difficult to include in the crawl lists as it is typically not readily available and there is not an established method to rank the social networking content either alone or with respect to a search engine. Search engines, in turn, have difficulty generating prioritized crawl lists that include content associated with social networking environments.